Parameter estimation apparatus and data matching apparatus

ABSTRACT

The present invention enables estimation of desired parameters with less computation cost and with high precision by inputting first training vectors generated from observation patterns and second training vectors generated from estimation targets in order to learn the correlation between observation patterns as inputs and patterns of the estimation targets such that desired outputs are assumed from the inputs, calculating the auto-correlation information of the two training vectors, and cross-correlation information of an average vector, the first training vectors and second training vectors, and using the information, obtaining probable expectation values based on the Bayes theory of the estimation targets with respect to an input pattern.

TECHNICAL FIELD

The present invention relates to a parameter estimation apparatus thatextracts specific parameters from an input pattern and a data matchingapparatus using the parameter estimation apparatus.

BACKGROUND ART

The processing for extracting specific parameters from an input patternis indeed general processing in pattern information processing, and forexample, includes processing for extracting positions of eyes and earsfrom an image of a human face and processing for extracting a positionof a number plate from an image of a vehicle.

Conventionally, the most popular method of such processing is called amatched filter method as summarized below, and an extremely large numberof applications have been proposed. As an example, a method ofextracting facial features will be described below with reference toFIG. 1.

As illustrated in the operation flow chart in FIG. 1, templates of eyeand ear regions are stored in template database 1601 in advance. Asillustrated in FIG. 2, a plurality of eye templates 1701 is stored intemplate database 1601.

When an input image is provided from a camera (S81), a single template1701 is obtained from template database 1601 (S82). Next, as illustratedin FIG. 3, input image 2001 is searched using search window 2002, andthe similarity degree between an image within search window 2002 andtemplate 1701 is obtained (S83). The computation of the similaritydegree usually uses the normalized correlation between the image withinsearch window 2002 and template 1701.

It is judged whether the above processing is executed on the whole inputimage 2001 (S84), input image 2001 is scanned using search window 2002until the scanning is performed on the whole input image 2001 (S85), andthe processing S83 is executed.

Then, it is judged whether the above search is performed with respect toall the templates 1701 contained in template database 1601 (S86). Whenthe processing is not executed with respect to all the templates 1701, atarget template 1701 is changed (S87), the processing flow shifts toS83, and the processing of S83 to S85 is executed on all the templates.

Based on similarity degrees between the image within search window 2002and templates 1701 obtained in the processing of S83 to S87, a positionof a local area (search window 2002 region) that is the most similar totemplate 1701 is found from input image 2001, and the positioncorresponding to the local area is output (S88).

An example of methods based on the aforementioned method is described indetail in R. Brunelli, T. Poggio, “Face recognition: Features Versustemplate”, IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-8, pages34 to 43, 1993.

A difficulty in the aforementioned conventional method is processingcost in computer. Assuming that a size of an input image in which searchis performed is S, template size is T, and the normalized correlation isused as a criterion of similarity degree, when the multiplication isunit computation, a time computation amount requires the number ofcomputations of 2×T×S. For example, in extracting coordinates of afeature point of a typical face image, under the assumption thatT=50×20=1000 (pel) and S=150×160=22500 (pel), it is required to multiply2×1000×22500=45×1000,000=4500 millions times. Such a large number ofmultiplications require enormous computation cost; even the computationspeed of a computer is improved.

Templates used in the processing usually use typical data such as anaverage of all learning data, which causes many cases that the matchingdoes not work well depending on environments. Therefore, there is amethod of performing the similarity degree computation using a pluralityof templates prepared corresponding to the input pattern. However, sucha method increases the number of processing corresponding to the numberof templates, and therefore, imposes loads on a computer also in term ofprocessing cost.

DISCLOSURE OF INVENTION

It is an object of the present invention to obtain coordinates offeature points of input data with reduced cost.

In the present invention, the correlation is learned in advance betweensample data and coordinates of a feature point in the data, and usingthe correlation obtained by the learning, coordinates of a feature pointof input data are estimated.

Since there is predetermined correlation between the same kind of dataand coordinates of a feature point in such data, it is possible toobtain coordinates of a feature point of the data with less processingcost and with accuracy, using the correlation obtained as describedabove.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an operation flow diagram of a conventional method ofextracting facial features;

FIG. 2 shows eye templates;

FIG. 3 is a view illustrating search by template;

FIG. 4 is a diagram illustrating a data matching apparatus according toa first embodiment of the present invention;

FIG. 5 is a diagram illustrating a learning section according to thefirst embodiment;

FIG. 6 is an operation flow diagram of offline processing of the datamatching apparatus according to the first embodiment;

FIG. 7 is a view illustrating coordinates of feature points of a faceinput by a user in the first embodiment;

FIG. 8 is a diagram illustrating a parameter estimating section in thefirst embodiment;

FIG. 9 is an operation flow diagram of online processing of the datamatching apparatus according to the first embodiment;

FIG. 10 is a view illustrating a face region detected in the firstembodiment;

FIG. 11 is a view output to a display in the first embodiment;

FIG. 12 is an operation flow diagram of offline processing of a datamatching apparatus according to a second embodiment of the presentinvention;

FIG. 13 is a view illustrating coordinates of feature points of a faceinput by a user in the second embodiment;

FIG. 14 is an operation flow diagram of online processing of the datamatching apparatus according to the second embodiment;

FIG. 15 is an operation flow diagram of offline processing of a datamatching apparatus according to a third embodiment of the presentinvention;

FIG. 16 is an operation flow diagram of online processing of the datamatching apparatus according to the third embodiment;

FIG. 17 is an operation flow diagram of offline processing of a vehicleimage matching apparatus according to a fourth embodiment of the presentinvention;

FIG. 18 is a view illustrating coordinates of feature points of avehicle image input by a user in the fourth embodiment;

FIG. 19 is an operation flow diagram of online processing of the vehicleimage matching apparatus according to the fourth embodiment;

FIG. 20 is a view output to a display in the fourth embodiment;

FIG. 21 is an operation flow diagram of offline processing of a datamatching apparatus according to a fifth embodiment of the presentinvention;

FIG. 22 is an operation flow diagram of online processing of the datamatching apparatus according to the fifth embodiment;

FIG. 23 is a diagram illustrating a learning section according to thefifth embodiment;

FIG. 24 is an operation flow diagram of offline processing of a datamatching apparatus according to a sixth embodiment of the presentinvention;

FIG. 25 is an operation flow diagram of online processing of the datamatching apparatus according to the sixth embodiment;

FIG. 26 is an operation flow diagram of offline processing of a datamatching apparatus according to a seventh embodiment of the presentinvention;

FIG. 27 is an operation flow diagram of online processing of the datamatching apparatus according to the seventh embodiment; and

FIG. 28 is an operation flow diagram of online processing of a vehicleimage matching apparatus according to an eighth embodiment of thepresent invention.

BEST MODE FOR CARRYING OUT THE INVENTION FIRST EMBODIMENT

The first embodiment of the present invention explains a case where aparameter estimation apparatus is applied to a data matching apparatus.

FIG. 4 is a diagram illustrating a data matching apparatus according tothe first embodiment of the present invention.

Image input section 101 converts optical data into electronic data tooutput.

Parameter input section 102 inputs coordinates of a feature point of thefirst electronic data input from image input section 101.

Learning section 103 calculates information on the first electronic datafrom the first electronic data input from image input section 101 andthe coordinates of a feature point input from parameter input section102.

Correlation information database 104 stores the information on the firstelectronic data calculated in learning section 103.

Parameter estimating section 105 estimates coordinates of a featurepoint in second electronic data from the second electronic input fromimage input section 101 and the information on the first electronic datastored in correlation information database 104.

Image database 106 stores the first electronic data.

Matching section 107 matches the first data with the second data toobtain the first data matching the coordinates of the feature point inthe second electronic data estimated in parameter estimating section105.

Output section 108 displays matching results matched in matching section107.

The data matching apparatus according to the first embodiment will bedescribed below. In the matching of an image of a face, it is importantto accurately obtain positions in which the face is present fromelectronic data input from image input section 101.

In the first embodiment, attention is drawn to the fact that there isthe correlation between an image of a face that is input electronic dataand coordinates of a feature point such as eyes, nose, eyebrows andmouth that are specific parameters in the image of the face. In otherwords, it is noted that when the correlation of coordinates ofparameters specific to face images are obtained in advance, it ispossible to obtain coordinates of feature points that are specificparameters when an unknown face image is input.

That is, in the first embodiment, the correlation is learned betweenface images (that are) samples and coordinates of feature points of thesample face images, and using the learned correlation, coordinates isestimated of feature points in a matching-target face image that is thesecond electronic data shot in image input section 101. Further, usingthe estimated coordinates of feature points, a face region that is aregion to match is obtained. Then, in the first embodiment, an image ofthe obtained face region is compared with each image stored beforehandin face image database 106, thus performing matching on face images.

Further, the first embodiment improves the accuracy in estimatingcoordinates of feature points in a matching-target face image, bylearning the correlation between face images samples and coordinates offeature points in the plurality of face images.

Specifically, (the) processing in the data matching apparatus in thefirst embodiment is divided roughly (broadly) into offline processingand online processing.

The offline processing calculates in advance the correlation between asample face images and coordinates of feature points of the face images.

The online processing estimates coordinates of feature points of amatching-target face image input from image input section 101 using thecorrelation calculated in the offline processing, and match an image ofthe face region determined from the coordinates of feature points withface images in image database 106 that are registered in advance by theoffline processing.

The offline processing will be described below. The offline processingis performed using image input section 101, learning section 103,correlation information database 104, and image database 106.

The offline processing performed in the data matching apparatus will bedescribed specifically below with reference to FIGS. 5 and 6. FIG. 5 isa block diagram illustrating a configuration of learning section 103.FIG. 6 is an operation flow diagram of the offline processing in thedata matching apparatus in the first embodiment.

A configuration of learning section 103 is explained first.

Learning section 103 is provided with second training vector calculatingsection 201 that calculates second training vectors obtained fromcoordinates of feature points in face images that are the firstelectronic data, and outputs the second training vectors.

Learning section 103 is further provided with first training vectorcalculating section 202 that calculates first training vectors obtainedfrom face images that are the first electronic data, and outputs thefirst training vectors.

Average vector calculating section 203 a calculates an average vector ofthe second training vectors calculated in second training vectorcalculating section 201. Average vector calculating section 203 bcalculates an average vector of the first training vectors calculated infirst training vector calculating section 202.

Cross-correlation information calculating section 204 calculates acovariance matrix that is the cross-correlation information between thefirst training vectors calculated in first training vector calculatingsection 202 and the second training vectors calculated in secondtraining vector calculating section 201.

Auto-correlation information calculating section 205 calculates a pseudoinverse matrix of the covariance matrix that is the auto-correlationinformation of the first training vectors obtained in first trainingvector calculating section 202.

Feature extraction matrix calculating section 206 calculates a featureextraction matrix from the pseudo inverse matrix of the covariancematrix of the auto-correlation information calculated in auto-correctioninformation calculating section 205 and the covariance matrix of thecross-correlation information calculated in cross-correlationinformation calculating section 204.

Next, the operation of the offline processing in the data matchingapparatus will be described with reference to FIG. 6.

Image input section 101 receives sample face images that is the firstelectronic data to input to parameter input section 102, learningsection 103 and image database 106 (S10).

In addition, it may be possible to enter electronic data different fromthe first electronic data to image database 106.

First training vector calculating section 202 in learning section 103converts respective values of pixels in an input sample face image intoa vector pattern where the values are arranged in the order of rasterscan to obtain training vector V₁ (S11) using equation 1 (S1), andoutputs the vector to vector average calculating section 203 b.V=[V ₁ ^(T) , V ₂ ^(T) , . . . , V _(M) ^(T)]^(T)  Eq.1[V ₁ , . . . , V _(M) are components of vector V (M≧1).]

A display included in parameter input section 102 displays a sample faceimage. When a user selects coordinates of feature points of a face fromthe face image using a mouse included in parameter input section 102,parameter input section 102 outputs the coordinates of feature points ofthe face selected by the user to leaning section 103 (S12).

The input coordinates of feature points of the face are shown in FIG. 7.Using origin 1208 of face image 1201 as the base, the user selectscoordinates (x-coordinate and y-coordinate) of each of right eyebrow1202, right eye 1203, left eyebrow 1204, left eye 1205, nose 1206 andmouth 1207, as coordinates of feature points.

Using Equation 1, second vector calculating section 201 in learningsection 103 arranges and combines input coordinates of feature points inthe order determined for the data matching apparatus, for example, inthe order of right eyebrow, right eye, left eyebrow, left eye, nose andmouth, to generate a single vector data as second training vector V₂(S13), and outputs the vector to average vector calculating section 203a.

Parameter input section 102, second training vector calculating section201 and first training vector calculating section 202 in learningsection 103 perform the processing of S10 to S13 repeatedly N timescorresponding to required N samples. When the processing is finished,the processing flow proceeds to S16. When the processing is notfinished, processing of S11 to S14 is repeated (S15).

Average vector calculating sections 203 a and 203 b calculate averagevector M₁ that is an average vector of training vectors V₁ and averagevector M₂ that is an average vector of training vectors V₂, using {V₁}that is a set of N training vectors V₁ and {V₂} that is a set of Ntraining vectors V₂ and Equations 2 and 3, respectively (16).

Then, average vector calculating section 203 b outputs average vector M₁to auto-correlation information calculating section 205 andcross-correlation information calculating section 206, and averagevector calculating section 203 a outputs average vector M₂ tocross-correlation information calculating section 204.

$\begin{matrix}{M_{1} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}\; V_{1\; i}}}} & {{Eq}.\mspace{14mu} 2} \\\begin{matrix}{{M_{2} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}\; V_{2\; i}}}}\mspace{31mu}} \\\begin{bmatrix}\begin{matrix}{M_{1}\mspace{14mu}{and}\mspace{14mu} M_{2}\mspace{14mu}{are}\mspace{14mu}{average}\mspace{14mu}{vectors}\mspace{14mu}{respectively}\mspace{14mu}{of}\mspace{11mu}\{ V_{1} \}\mspace{14mu}{and}\mspace{14mu}\{ V_{2} \}} \\{N\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{number}\mspace{14mu}{of}\mspace{14mu}{samples}}\end{matrix} \\{i\mspace{14mu}{is}\mspace{14mu}{an}\mspace{14mu}{index}\mspace{14mu}{to}\mspace{14mu}{each}\mspace{14mu}{{{one}{\;\;}( {1 \leq i \leq N} )}.}}\end{bmatrix}\end{matrix} & {{Eq}.\mspace{14mu} 3}\end{matrix}$

Auto-correlation information calculating section 205 substitutes {V₁}that is set of N training vectors V₁ and average vector M₁ into Eq.4,calculates covariance matrix C₁ that is the distribution of trainingvector V₁, and thus calculates the distribution of N face images inputfrom image input section 101. Auto-correlation information calculatingsection 205 transforms covariance matrix C₁ into pseudo inverse matrixC₁* to output to feature extraction matrix calculating section 206.

$\begin{matrix}\begin{matrix}{C_{1} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{( {V_{1\; i} - M_{1}} )( {V_{1\; i} - M_{1}} )^{T}}}}} \\\begin{bmatrix}\begin{matrix}{{M_{1}\mspace{14mu}{is}\mspace{14mu}{the}{\mspace{11mu}\mspace{11mu}}{a{verage}}\mspace{14mu}{vectors}\mspace{14mu}{of}\{ V_{1} \}}\mspace{11mu}} \\{N\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{number}\mspace{14mu}{of}\mspace{14mu}{samples}}\end{matrix} \\{i\mspace{14mu}{is}\mspace{14mu}{index}\mspace{14mu}{to}\mspace{14mu}{each}\mspace{14mu}{one}\;{( {1 \leq i \leq N} ).}}\end{bmatrix}\end{matrix} & {{Eq}.\mspace{14mu} 4}\end{matrix}$

Further, cross-correlation information calculating section 204substitutes N training vectors V₁, N training vectors V₂, average vectorM₁, and average vector M₂ into Equation 5, calculates covariance matrixC₂ that is the correlation between training vectors V₁ and trainingvectors V₂, thus calculates the correlation between the N face imagesand coordinates of feature points in the face images to output tofeature extraction matrix calculating section 206 (S17).

$\begin{matrix}\begin{matrix}{C_{2} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{( {V_{2\; i} - M_{2}} )( {V_{1\; i} - M_{1}} )^{T}}}}} \\\begin{bmatrix}\begin{matrix}{M_{1}\mspace{14mu}{and}\mspace{20mu} M_{2}\mspace{14mu}{are}\mspace{14mu}{average}\mspace{14mu}{vectors}\mspace{14mu}{respectively}\mspace{14mu}{of}\;\{ V_{1} \}\mspace{11mu}{and}\mspace{11mu}\{ V_{2} \}} \\{N\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{number}\mspace{14mu}{of}\mspace{14mu}{samples}}\end{matrix} \\{i\mspace{14mu}{is}\mspace{14mu}{an}\mspace{14mu}{index}\mspace{14mu}{to}\mspace{14mu}{each}\mspace{14mu}{one}\;{( {1 \leq i \leq N} ).}}\end{bmatrix}\end{matrix} & {{Eq}.\mspace{14mu} 5}\end{matrix}$

Next, feature extraction matrix calculating section 206 calculatesfeature extraction matrix C₃ for extracting coordinates of featurepoints from pseudo inverse matrix C₁* and covariance matrix C₂ obtainedin S17 and the matching-target face image input in the onlineprocessing, according to Equation 6 (S18).C ₃ =C ₂ C ₁*  Eq.6

Then, feature extraction matrix calculating section 206 outputs obtainedaverage vectors M₁ and M₂ and feature extraction matrix C₃ tocorrelation information database 104 (S19). Thus, the offline processingis finished.

The online processing will be described below with reference to FIGS. 8and 9. FIG. 8 is a diagram illustrating a configuration of parameterestimating section 105. FIG. 9 is an operation flow diagram of theonline processing of the data matching apparatus according to the firstembodiment.

Parameter estimating section 105 is provided with vector calculatingsection 301 that calculates an input vector X from the second electronicdata to output.

Parameter calculating section 302 calculates an expectation vectorcontaining coordinates of feature points from input vector X calculatedin parameter estimating section 105, average vectors M₁ and M₂ andfeature extraction matrix C₃ to output.

Template matching section 303 calculates coordinates of feature pointsfrom the expectation vector calculated in parameter calculating section302 to output.

A purpose of the online processing is estimating coordinates of featurepoints that are parameters specific to the input from the input faceimage, obtaining a face region that is a region image of a face used inmatching from the estimated coordinates of feature points, and collatingthe obtained face region with each image registered with image database106.

First, image input section 101 receives as its input a matching-targetimage that is the second electronic data (S20) to output to vectorcalculating section 301 in parameter estimating section 105. Vectorcalculating section 301 arranges respective values of pixels in thesecond electronic data in the order of raster scan to convert into aninput vector data X (S21), and outputs the data to parameter calculatingsection 302.

Parameter calculating section 302 substitutes the input vector data X,and feature extraction matrix C₃ and average vectors M₁ and M₂ obtainedin the offline processing into Equation 7, and calculates theexpectation vector E containing coordinates of feature points withrespect to the input vector data X (S22) to output to template matchingsection 303.E=M ₂ +C ₃(X−M ₁)  Eq.7

Equation 7 is to calculate coordinates of feature points of thematching-target image using the cross-correlation between N face imagesand coordinates of feature points of the face images obtained in theoffline processing.

As in calculating the computation cost in template matching, thecomputation cost in the present invention is calculated using the sameexample. It is assumed that an input image size is 150 in vertical×150in horizontal pixels (22500 pixels) and coordinates of both eyes(x-coordinate and y-coordinate of the right eye and x-coordinate andy-coordinate of the left eye) i.e. four dimensions are estimated withrespect to the input. In this case, when the vector where the inputimage of 150×150 is converted into a vertical line is substituted intoEquation 7 and multiplication is unit calculation, since C₃ is a matrixof 22500 in vertical×4 in horizontal and (X−M₁) is a vector of 22500dimensions, the number of multiplications is 4×22500=90000 (pel). Inother words, even only in the multiplication, the computation cost is1/500 that in template matching, thus resulting in great effects.

The reason for calculation using Equation 7 to enable estimation ofcoordinates of feature points is explained herein.

Expectation vector E indicative of coordinates of feature pointsobtained in Equation 7 is equal to an expectation of an output withrespect to the input vector data X obtained by learning the relationshipbetween training vectors V₁ and V₂ using the Bayes estimation on theassumption that the distributions of two vectors are normaldistributions.

The Bayes estimation is a statistical estimation method for defining thedistribution of population parameters and proper loss function andestimating so as to minimize the expectation of the loss function. Inother words, it is understood that using Equation 7, it is possible toestimate the most likely output value with respect to the input vectordata X.

Template matching section 303 calculates coordinates of feature pointsof the input vector data from the excitation vector E that is a combinedvector of coordinates of feature points (S23), and outputs thecoordinates of feature points to matching section 107.

Using the coordinates of feature points obtained in S23, matchingsection 107 determines a face region that is an image for use inmatching with face images registered in the offline processing withimage database 106.

FIG. 10 shows an example of detected face region for matching. In thisembodiment, a square region such that one side is twice the length of aninterval “a” between both eyes and upper and lower sides are parallel tothe straight line connecting both eyes is determined as matching region1302 that is a region for use in matching.

Matching section 107 matches the image of face region 1302 with imagesregistered in the offline processing in advance with image database 106,using a matching method, for example, the Eigenface method using theprincipal component analysis that is a statistical method (S25), andoutputs results to output section 108 (S26).

In addition, in the Eigenface method, sizes of reference images arenormalized, gray levels of all the pixels are set as an N-dimensionalvector, information amounts are compressed as F-dimensional information(0<F<N) from all the reference images, and a face portion space isgenerated using the statistical method called the principal componentanalysis. Then, a region with the high probability of presence of a faceis normalized from an input image, the orthogonal length to the faceportion space is set as the similarity degree, and a person isrecognized from a position of the projection on the face portion space.

FIG. 11 shows examples of outputs in output section 108. Display 1501displays input face image 1502 and matching results 1504 and 1505 inputto the display.

As described above, according to the first embodiment, using thecorrelation between sample face images that is the first electronic datalearned beforehand in the offline processing and coordinates of featurepoints in the sample face images, it is possible to estimate coordinatesof feature points in a matching-target face image that is the secondelectronic data shot in the image input section 101.

Then, it is possible to obtain a face region for matching from thecoordinates of feature points estimated as described above, and comparean image of the face region for matching with face images registeredbeforehand in the offline processing with image database 106 to match.

Further, it is possible to estimate parameters specific to an inputvector by three matrix manipulations. In other words, since it ispossible to estimate the parameters with extremely smaller cost thanthat in search using the conventional template matching, it is possibleto estimate required parameters with extremely less manipulation thanthat in the conventional method and to match face images with lessmanipulation.

Furthermore, according the first embodiment, the auto-correlationinformation is obtained from the covariance matrix of the first trainingvectors, the cross-correlation information is obtained from thecovariance matrix of the first and second training vectors, the featureextraction matrix is obtained from the auto-correlation information andcross-correlation information, and coordinates of feature points of amatching-target face image are calculated from the average vector of thefirst training vectors, the average vector of the second trainingvectors, the feature extraction matrix and input vector data. Thus,since it is possible to calculate the auto-correlation information,cross-correlation information, feature extraction matrix, the averagevector of the first training vectors, the average vector of the secondtraining vectors, and coordinates of feature points of a matching-targetface image simply by matrix manipulation i.e. only by matrixmultiplication, the processing cost of computer becomes extremely small.

In addition, while in the first embodiment parameters are estimatedusing above-mentioned Equations 1 to 7, same effects may be obtainedusing equations other than Equations 1 to 7, as long as the equationsare to beforehand obtain correlation between sample face images andcoordinates of feature points in the face images, while estimatingparameters using the obtained correlation.

Further, it may be possible to estimate coordinates of feature pointsother than the first electronic data that is sample image data.

SECOND EMBODIMENT

In contrast to the first embodiment where the training vector V₂ isobtained from respective coordinates of feature points of a face, thesecond embodiment describes obtaining the training vector V₂ from regionimages around the coordinates of feature points of a face, andautomatically generating a template suitable for a matching-target faceimage from the matching-target face image. It is thereby possible toexecute matching processing using only a sheet of template, and toreduce the computation cost greatly as compared with the conventionalcase of using a plurality of templates.

Specifically, in the second embodiment, the correlation is examinedbetween sample face images that are the first electronic data shot inadvance in image input section 101 and region images that are imagesaround coordinates of feature points such as eyes, nose, mouth andeyebrows in the sample face images. Then, in the second embodiment,using the correlation, a specific region image for use in matching isobtained as a template from the matching-target face image that is thesecond electronic data shot in image input section 101.

Next, in the second embodiment, the matching is performed between theobtained template and the matching-target face image, coordinates offeature points of the matching-target face image are thereby estimated,and a face region image is obtained that is a region for use inmatching.

In the second embodiment, the face region image is compared with eachimage in image database prepared in advance, and thereby matching isperformed on the face image that is the second electronic data.

A data matching apparatus according to the second embodiment will bedescribed specifically below.

A diagram illustrating a configuration of the data matching apparatusaccording to the second embodiment is the same as that of the datamatching apparatus in the first embodiment, and descriptions thereof areomitted.

As in the first embodiment, the processing in the data matchingapparatus according to the second embodiment is divided broadly intooffline processing and online processing. Herein, differences from thefirst embodiment are particularly explained with reference to FIGS. 12to 14.

FIG. 12 is an operation flow diagram illustrating the offline processingaccording to the second embodiment, FIG. 13 is a view illustratingregion images that are images around respective coordinates of featurepoints from a sample face image that is the first electronic data, andFIG. 14 is an operation flow diagram illustrating the online processing.

In the offline processing in the second embodiment, the processing ofS90 to S92 is the same as that of S10 to S12 illustrated in FIG. 6, theprocessing of S94 to S98 is the same as that of S15 to S19 illustratedin FIG. 6, and descriptions thereof are omitted.

Second training vector calculating section 201 in learning section 103selects region images that are images around respective coordinates offeature points as illustrated in FIG. 13 from coordinates of featurepoints selected in S92.

Using origin 1402 of face image 1401 as the base, coordinates(x-coordinate and y-coordinate) of each of right eyebrow 1403 a, righteye 1404 a, left eyebrow 1405 a, left eye 1406 a, nose 1407 a and mouth1408 a are input to second training vector calculating section 201, ascoordinates of feature points.

Second training vector calculating section 201 generates right eyebrowregion image 1403 b, right eye region image 1404 b, left eyebrow regionimage 1405 b, left eye region image 1406 b, nose region image. 1407 band mouth region image 1408 b that are template images with respectivecoordinates of feature points of a face from input coordinates offeature points 1403 a to 1408 a.

As a method of determining each region image as a rectangle, using theinterval “a” between both eyes obtained in FIG. 10 as the base, thewidth and height of each region are determined, and a region is setusing the coordinate of a feature point as a center. The width andheight of an eyebrow are a/2 and a/4, width and height of an eye is a/2and a/4, width and height of the nose is “a” ×⅔ and “a” ×⅔, and widthand height of the mouth is “a” and a/2, respectively.

Thus, by using the interval “a” between both eyes as the base, it ispossible to determine each region image independently of the size of aface indicated in the face image in FIG. 13.

Second training vector calculating section 201 rearranges values ofpixels in generated template images 1403 b to 1408 b with respectivecoordinates of feature points in the order of raster scan using Equation1, and generates a combined vector with vector data as training vectorV₂ (93).

In addition, feature extraction vector C₃ obtained in S97 is generatedusing training vector V₂, and therefore, are parameters to extract aregion image.

The online processing according to the second embodiment will bedescribed with reference to FIG. 14. The processing of S110 to S112 isthe same as that of S20 to S22 in FIG. 9, and descriptions thereof areomitted.

Expectation vector E generated in S112 is a combined vector of templateimages of respective coordinates of feature points in thematching-target face image that is the second electronic data usingaverage vectors M₁ and M₂ respectively of training vectors V₁ and V₂obtained in S95 in the offline processing, feature extraction matrix C₃obtained in S97 in the offline processing and vector X obtained in S111in the online processing. Therefore, parameter calculating section 302obtains template images with respective coordinates of feature points ofthe face from the expectation vector E (S113).

Template matching section 303 performs template matching betweentemplate images obtained in S113 and the matching-target image inputfrom image input section 101 obtained in S110, detects regions matchingthe template images, and from the detected regions, determinesrespective coordinates of feature points of the face (S114).

In addition, the template matching processing is processing for scanningon input image 2001 with search window 2002 as illustrated in FIG. 3,and selecting a region with the highest correlation with the templateimage when an image in the search window is input. The correlationcomputation used in the template matching uses, for example, SADcorrelation computation (Eq.8) and normalized correlation computation(Eq.9). In addition, in Eq.8 and Eq.9, X is an image in a search windowand T is a template image.

SAD Correlation Computation

$\begin{matrix}\begin{matrix}{C_{SAD} = {\sum\limits_{i = 1}^{N}{{X_{i} - T_{i}}}}} \\\begin{bmatrix}\begin{matrix}\begin{matrix}{C_{SAD}\mspace{14mu}{is}\mspace{14mu}{an}\mspace{14mu}{SAD}\mspace{14mu}{correlation}\mspace{14mu}{value}} \\{X_{i}\mspace{14mu}{is}\mspace{14mu}{an}\mspace{14mu}{ith}\mspace{14mu}{pixel}\mspace{14mu}{value}\mspace{14mu}{of}\mspace{14mu}{input}\mspace{14mu}{vector}}\end{matrix} \\{{T_{i}\mspace{14mu}{is}\mspace{14mu}{an}\mspace{14mu}{ith}\mspace{14mu}{pixel}\mspace{14mu}{value}\mspace{14mu}{of}\mspace{14mu} a\mspace{14mu}{template}}\mspace{14mu}}\end{matrix} \\{N\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{number}\mspace{14mu}{of}\mspace{14mu}{pixels}\mspace{14mu}{included}\mspace{14mu}{in}\mspace{14mu}{the}\mspace{14mu}{{template}.}}\end{bmatrix}\end{matrix} & {{Eq}.\mspace{14mu} 8}\end{matrix}$Normalized Correlation Computation

$\begin{matrix}\begin{matrix}{C_{norm} = \frac{\sum\limits_{i = 1}^{N}\;{( {X_{i} - \overset{\_}{X}} ) \cdot ( {T_{i} - \overset{\_}{T}} )}}{\sqrt{\sum\limits_{i = 1}^{N}\;( {X_{i} - \overset{\_}{X}} )^{2}} \times \sqrt{\sum\limits_{i = 1}^{N}\;( {T_{i} - \overset{\_}{T}} )^{2}}}} \\\begin{bmatrix}\begin{matrix}\begin{matrix}\begin{matrix}\begin{matrix}{{C_{norm}\mspace{14mu}{is}\mspace{14mu} a\mspace{14mu}{normalized}\mspace{14mu}{correlation}\mspace{14mu}{value}}\mspace{14mu}} \\{X_{i}\mspace{11mu}{is}\mspace{14mu}{an}\mspace{14mu}{ith}\mspace{14mu}{value}\mspace{14mu}{of}\mspace{14mu}{input}\mspace{14mu}{vector}}\end{matrix} \\{T_{i}\mspace{14mu}{is}\mspace{14mu}{an}\mspace{14mu}{ith}\mspace{14mu}{value}\mspace{14mu}{of}\mspace{14mu}{template}}\end{matrix} \\{\overset{\_}{X}\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{average}\mspace{14mu}{of}\mspace{14mu}{input}\mspace{14mu}{vectors}}\end{matrix} \\{{\overset{\_}{T}\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{average}\mspace{14mu}{of}\mspace{14mu}{templates}\mspace{14mu}{and}}\mspace{14mu}}\end{matrix} \\{N\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{number}\mspace{14mu}{of}\mspace{14mu}{pixels}\mspace{14mu}{included}\mspace{14mu}{in}\mspace{14mu}{the}\mspace{14mu}{{template}.}}\end{bmatrix}\end{matrix} & {{Eq}.\mspace{14mu} 9}\end{matrix}$

Subsequent processing, S115 to S117, is the same as that of S24 to S26in FIG. 9, and descriptions thereof are omitted.

As described above, according to the second embodiment, the correlationis learned between sample images and specific region images of thesample images, specific region images of a matching-target face imageare estimated using the correlation, and estimated specific regionimages of the matching-target face image are set as template images. Inthis way, since template images can be generated automatically from thematching-target face image itself, a sheet of template allows thematching processing, and it is possible to greatly reduce thecomputation cost as compared to the conventional case of using aplurality of templates.

Further, according to the second embodiment, it is possible to calculatecoordinates of feature points of a matching-target face image from thefeature extraction matrix obtained from the auto-correlation informationand cross-correlation information of the first training vectors andsecond training vectors, the average vector of the first trainingvectors, the average vector of the second training vectors, and thevector data of the matching-target face image. Thus, since specificparameters can be calculated simply by matrix manipulation i.e. only bymultiplication, the processing cost of computer becomes extremely small.

Furthermore, in the case of applying the second embodiment of thepresent invention to template image estimation in template matching, itis made possible to estimate a specific image expected from an inputimage, and perform matching processing using the specific image as atemplate. In other words, since the processing can be executed using atemplate of one pattern, it is possible to reduce calculation cost ascompared to a multi-template method of preparing a plurality oftemplates different in patterns and sizes corresponding to versatilityof input images like the conventional template matching.

THIRD EMBODIMENT

The third embodiment of the present invention describes obtaining thetraining vector V₂ from combined vectors of respective coordinates offeature points of a face and region images around the respectivecoordinates of feature points, in contrast to the first embodiment wherethe training vector V₂ is obtained from respective coordinates offeature points of a face and to the second embodiment where the trainingvector V₂ is obtained from region images around respective coordinatesof feature points of a face.

Then, coordinates of feature points of a matching-target face image areestimated from thus obtained training vectors V₂, a template suitablefor the matching-target face image is automatically generated from thematching-target face image, and using the automatically generatedtemplate with peripheral regions of coordinates of feature points,matching is performed.

It is thereby possible to limit a search range of the template, andperform the matching processing using a sheet of template. As a result,it is possible to greatly reduce cost as compared to the case ofsearching the entire region of a matching-target face image and using aplurality of templates as in the conventional method.

A data matching apparatus according to the third embodiment will bedescribed specifically below.

A diagram illustrating a configuration of the data matching apparatusaccording to the third embodiment is the same as that of the datamatching apparatus in the first embodiment, and descriptions thereof areomitted.

As in the first embodiment, the processing of the data matchingapparatus according to the third embodiment is divided broadly intooffline processing and online processing. Herein, particularly,differences from the first and second embodiments are explained withreference to FIGS. 15 and 16.

FIG. 15 is an operation flow diagram illustrating the offline processingaccording to the third embodiment, and FIG. 16 is an operation flowdiagram illustrating the online processing.

In the offline processing in the second embodiment, processing of S1200to S1202 is the same as that of S10 to S12 shown in FIG. 6, processingof S1205 to S1209 is the same as that of S15 to S19 shown in FIG. 6, anddescriptions thereof are omitted.

In S1202, parameter input section 102 selects region images that areimages around coordinates of feature points as shown in FIG. 13 fromselected coordinates of feature points (S1203).

Using Eq.1, learning section 103 generates a combined vector of vectordata where respective coordinates of facial feature points are arrangedsequentially and combined and another vector data where respective pixelvalues constituting generated template images 1403 b to 1408 b with therespective coordinates of facial feature points are arranged in theorder of rater scan, as training vector V₂ (S1204).

In addition, feature extraction vector C₃ obtained in S1208 is generatedusing training vector V₂, and therefore, are parameters to extractcoordinates of feature points and region images.

The online processing will be described with reference to FIG. 16. Theprocessing of S1300 to S1303 is the same as that of S20 to S22 in FIG.9, the processing of S1306 to S1308 is the same as that of S24 to S26 inFIG. 9, and descriptions thereof are omitted.

Expectation vector E generated in S1302 is a combined vector of templateimages of respective coordinates of feature points in thematching-target face image that is the second electronic data usingaverage vectors M₁ and M₂ respectively of training vectors V₁ and V₂obtained in offline processing S1206, feature extraction matrix C₃, forextracting coordinates of feature points and region images obtained inS1208, and vector X obtained in S1301 in the online processing.Therefore, parameter calculating section 302 obtains coordinates offeature points in the matching-target face image in S1303, and templateimages that are region images in S1304, from the expectation vector E.

Template matching section 303 performs template matching on theperipheries of coordinates of feature points between template imagesobtained in S1304 and the matching-target image input from image inputsection 101 obtained in S1300, detects regions matching the templateimages, and from the detected regions, determines respective coordinatesof feature points of the face (S1305).

As in the first embodiment, the correlation computation used in thetemplate matching uses SAD correlation computation (Eq.8) and normalizedcorrelation computation (Eq.9), for example.

As described above, according the third embodiment, a combined vector isestimated from respective coordinates of feature points in amatching-target face image and region images of the respectivecoordinates of feature points, and a search range is limited to theperipheries of estimated respective coordinates of feature points of theface.

Further, hierarchical search can be performed such that templatematching search is performed on the limited search range, using regionimages of estimated respective coordinates of feature points astemplate.

In this way, the detection precision is improved due to the hierarchicalsearch, while the search range is decreased, and the template to use isonly a single sheet of estimated one, enabling detection with lesscomputation cost than in the conventional case.

Further, according to the third embodiment, the auto-correlation isobtained by calculating the covariance matrix of the first trainingvector, and the cross-correlation is obtained by calculating thecovariance matrix of the first and second training vectors. Thus, theauto-correlation and cross-correlation can be obtained simply by matrixmanipulation i.e. only by multiplication, and therefore, the processingcost becomes extremely small.

FOURTH EMBODIMENT

The fourth embodiment of the present invention provides a vehicle imagematching apparatus where a parameter estimation apparatus is applied tomatching of vehicle images.

The vehicle image matching apparatus detects coordinates of featurepoints such as a number plate, lights and driver sheet from images ofcontrast pattern or the like including a vehicle on a road shot in imageinput section 101, using the Bayes estimation. Then, the vehicle imagematching apparatus matches data to check whether the image of the entirevehicle obtained based on obtained coordinates of feature points andfeatures such that numbers of the number plate match those of a vehicleregistered in advance.

A diagram illustrating a configuration of the vehicle image matchingapparatus is the same as FIG. 4 in the first embodiment, anddescriptions thereof are omitted.

The processing in the vehicle image matching apparatus is dividedbroadly into offline processing and online processing. Herein,particularly, differences from the first embodiment are explained withreference to FIGS. 17 to 19.

FIG. 17 is an operation flow diagram illustrating the offline processingaccording to the fourth embodiment.

The processing of S1401 and S1405 to S1408 in FIG. 17 is the same asthat the processing of S11 and S16 to S19 in FIG. 6 in the firstembodiment, and descriptions thereof are omitted.

Image input section 101 inputs N vehicle images for use in matching toparameter input section 102, learning section 103 and image database, asinput first electronic data (S1400).

First training vector calculating section 202 in learning section 103calculates first training vector V₁ (S1401).

A display included in parameter input section 102 displays coordinatesof feature points of the vehicle image on a sheet basis. When a userselects coordinates of feature points of the vehicle image using a mouseincluded in parameter input section 102, parameter input section 102outputs selected coordinates of feature points to leaning section 103(S402).

FIG. 18 shows coordinates of feature points of the vehicle image. Ascoordinates of feature points of the vehicle in input image 1701, frontportion 1708 and coordinates A 1702 to coordinates F 1707 that arevertexes of the vehicle are input.

Second vector calculating section 210 in learning section 103 calculatessecond training vector V₂ from the input coordinates of feature points(S1403).

Parameter input section 102, second training vector calculating section201 and first training vector calculating section 202 in learningsection 103 determine whether the processing corresponding to N vehiclesis finished (S1404), and when the processing is finished, the processingflow proceeds to S1405, while shifting to S1400 when the processing isnot finished.

Subsequent processing, S1405 to S1408, is the same as that of S16 to S19shown in FIG. 6 i.e. the same as in the first embodiment, anddescriptions thereof are omitted.

The online processing will be described below with reference to theoperation flow shown in FIG. 19.

A purpose of the online processing is estimating coordinates of requiredfeature points from the input vehicle image, and using the coordinatesof feature points, analyzing and storing the vehicle image and images ofthe number plate and front portion.

The processing of S1900 to S1903 and S1906 in FIG. 19 is the same asthat of S20 to S23 and S26 as in the first embodiment, and descriptionsthereof are omitted.

Matching section 107 determines an interval as the base from coordinatesof feature points obtained in S1903, and based on the interval, andfetches the vehicle image, region image of the number plate portion andregion image of the front portion (S1904).

Then, matching section 107 reads numbers on the number plate of thevehicle, for example, by scanning the region image of the number plateportion using a scanner (S1905).

Further, matching section 107 matches the vehicle image and number platewith images registered in advance in the offline processing with imagedatabase 106, using a matching method, for example, the Eigenface method(S1906).

FIG. 20 shows display examples of input image 1802, vehicle image 1804,region image of number plate portion and scanned result 1805, matchingresult 1806, and region image of front portion 1807.

As described above, according to the fourth embodiment, by graphicallyshowing an input vehicle image and estimated coordinates of featurepoints, scanning the image of the entire vehicle, the image of thenumber plate portion, and numbers of the number plate portion based onthe coordinates of feature points, and displaying results from collatingwith vehicles registered in advance and enlarged images of the driverseat and passenger seat, it is possible to readily understand theinformation of the vehicle and state of the driver.

In addition, while in the fourth embodiment the parameter estimatingmethod in the first embodiment is used, it may be possible to apply thesecond or third embodiment to a vehicle image matching apparatus.

FIFTH EMBODIMENT

The fifth embodiment of the present invention copes with inputmatching-target images which have various distributions whenfluctuations exist on the distributions of input vectors that are theinput sample images.

Specifically, combined information of input sample images andcoordinates of feature points are divided into a plurality ofdistributions, correlation between a matching-target image andcoordinates of feature points of the matching-target images is studiedfor each distribution, and using the correlations, coordinates offeature points of the matching-target image are obtained.

Further, the processing of data matching apparatus according to thefifth embodiment is divided broadly into offline processing and onlineprocessing, as in the first embodiment. Herein, differences from thefirst embodiment are particularly explained with reference to FIGS. 21to 23.

A diagram illustrating a configuration of the data matching apparatusaccording to the fifth embodiment is the same as that of the datamatching apparatus in the first embodiment, and descriptions thereof areomitted, except learning section 103 as illustrated in FIG. 23. “401”denotes a distribution element calculating section that classifies aplurality of first training vectors and second training vectors intoelement distributions based on the probability distribution.

FIG. 21 is a view illustrating an operation flow of the offlineprocessing according to the fifth embodiment, and FIG. 22 is a viewillustrating an operation flow of online processing.

The processing of S2100 to S2104 in the offline processing is the sameas that of S10 to S15 illustrated in FIG. 6 as in the first embodiment,and descriptions thereof are omitted.

After confirming input of N items of first electronic data in S2104,distribution element calculating section 401 constitutes combinedvectors of training vectors V₁ and V₂ of N people. Then, distributionelement calculating section 401 models the probability distribution in aset of combined vectors of N people to element distributions with Rdistributions using Gaussian mixed model (hereafter referred to as GMM),and outputs the first training vector and second training vector foreach element distribution to average calculating section 203 a.

Average vector calculating section 203 a, average vector calculatingsection 203 b, cross-correlation information calculating section 204 andauto-correlation information calculating section 205 calculateparameters of kth (k=1, . . . , R) element distribution in modeling.

In other words, average vector calculating section 203 b calculates theaverage vector M₁ ^(k) of vector V₁ belonging to the kth elementdistribution, average vector calculating section 203 a calculates theaverage vector M₂ ^(k) of vector V₂ belonging to the kth elementdistribution, the auto-correlation information calculating sectioncalculates C₁ ^(k)* that is the pseudo inverse matrix of the covariancematrix C₁ ^(k) of vector V₁, and the cross-correlation informationcalculating section calculates cross-correlation matrix C₁₂ ^(k) ofvectors V₁ and V₂ (S1205).

Average vector calculating section 203 b outputs the average vector M₁^(k) to auto-correlation information calculating section 205 andcross-correlation information calculating section 204, and averagevector calculating section 203 a outputs the average vector M₂ ^(k) tocross-correlation information calculating section 204.

Generally, the EM (Expectation Maximization) algorithm is used for thecalculation, which is described in detail in Christopher M. Bishop,Oxford, “Neural Networks for Pattern Recognition” pages 59 to 73 (1995).

Auto-correlation information calculating section 206 calculates featureextraction matrix C₃ ^(k) from obtained pseudo inverse matrix C₁ ^(k)*of the covariance matrix of vector V₁ and cross-correlation matrix C₁₂^(k), according to Equation 10 (S2106).C ₃ ^(K) =C ₁₂ ^(K) C ₁ ^(K)*  Eq.10C₁ ^(K)* is the pseudo inverse matrix of matrix C₁ ^(k).

Feature extraction matrix calculating section 206 outputs averagevectors M₁ ^(k) and M₂ ^(k) and feature extraction matrix C₃ ^(k) to acorrelation information database (S1207).

When average vectors M₁ ^(k) and M₂ ^(k) and feature extraction matrixC₃ ^(k) of each of R distributions are stored in the correlationinformation database, learning section 103 finishes the offlineprocessing. Meanwhile, when average vectors M₁ ^(k) and M₂ ^(k) andfeature extraction matrix C₃ ^(k) of each of R distributions are notstored in the correlation information database, learning section 103repeats the processing of S2105 to S2107 (S2108).

The online processing in the fifth embodiment will be described belowwith reference to the operation flow diagram shown in FIG. 22.

The processing of S2200, S2201 and S2203 to S2206 in the onlineprocessing in the fifth embodiment is the same as that of S20, S21 andS22 to S26 in FIG. 9 as in the first embodiment, and descriptionsthereof are omitted.

With respect to input vector data X calculated in S2201, parametercalculating section 302 calculates an expectation vector E ofcoordinates of feature points with respect to the input vector data Xfrom feature extraction matrix C₃* and average vectors M₁ and M₂obtained in the offline according to Equation 11 (S2202).

$\begin{matrix}{\begin{matrix}{\mspace{20mu}{E = {\frac{\sum\limits_{K = 1}^{R}\;{P( {w_{K}{ X )\lbrack {M_{2}^{K} + {C_{3}^{K}( {X - M_{1}^{K}} )}} \rbrack}} }}{\sum\limits_{K = 1}^{M}\;{P( {w_{K} X )} }}\mspace{25mu}{where}}}} \\{\mspace{20mu}{P( {{w_{K} X )} \equiv \frac{P_{K}{C_{1}^{K}}^{- \frac{1}{2}}\exp\{ {{- \frac{1}{2}}( {X - M_{1}^{K}} )^{T}{C_{1}^{K*}( {X - M_{1}^{K}} )}} \}}{\sum\limits_{l = 1}^{R}\;{P_{l}{C_{1}^{l}}^{- \frac{1}{2}}\exp\{ {{- \frac{1}{2}}( {X - M_{1}^{l}} )^{T}{C_{1}^{l*}( {X - M_{1}^{l}} )}} \}}}} }}\end{matrix}{{{P_{K}\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{prior}\mspace{14mu}{probability}\mspace{14mu}{of}\mspace{14mu}{Kth}\mspace{14mu}{element}\mspace{14mu}{distribution}}{of}\mspace{14mu}{mixed}\mspace{14mu}{distribution}\mspace{14mu}{modeled}\mspace{14mu}{from}\mspace{14mu}{combine}\mspace{14mu}{probability}}{distributions}\mspace{14mu}{of}\mspace{14mu}{the}\mspace{14mu}{first}\mspace{14mu}{and}\mspace{14mu}{second}\mspace{14mu}{training}\mspace{14mu}{vectors}}{R\mspace{14mu}{is}\mspace{14mu}{the}\mspace{14mu}{number}\mspace{14mu}{of}\mspace{14mu}{elements}\mspace{14mu}{in}\mspace{14mu}{the}\mspace{14mu}{mixed}\mspace{14mu}{distribution}}} & {{Eq}.\mspace{14mu} 11}\end{matrix}$

Expectation vector E of coordinates of feature points obtained inEquation 11 is equal to an expectation of an output with respect to theinput vector data X obtained by learning the relationship betweentraining vectors V₁ and V₂ using the Bayes estimation on the assumptionthat the distribution of combined vector of two vectors is mixed normaldistribution. The Bayes estimation is a statistical estimation methodfor defining the distribution of population parameters and proper lossfunction and estimating to minimize the expectation of the lossfunction. In other words, it is understood that using Equation 11, it ispossible to estimate the most likely output value with respect to theinput vector data X.

The subsequent processing, S2203 to S2206, is the same as in the firstembodiment, and descriptions thereof are omitted.

As described above, according to the fifth embodiment, input sampleimages are divided into a plurality of distributions, correlationbetween input sample images and coordinates of feature point of theinput images are studied for each distribution, and using thecorrelation, coordinates of feature points of an input matching-targetimage are estimated. It is thereby possible to accurately estimate thecoordinates of feature points even when fluctuations exist ondistributions i.e. characteristics of input matching-target images.

Further, as indicated in Eq.11, it is possible to estimate specificparameters of an input vector in direct calculation by matrixmanipulation. In this way, it is possible to perform estimation withgreatly less cost than the search by repeat computations using theconventional matching and with greatly high accuracy because the mixeddistribution model is used, thus resulting in effectiveness of extremelyhigh degree.

Furthermore, according to the fifth embodiment, it is possible to obtaincross-correlation information for each element distribution simply bymatrix manipulation i.e. only by multiplication.

SIXTH EMBODIMENT

The sixth embodiment copes with the case where fluctuations exist on thedistributions of input vectors that are input sample images. In contrastto the fifth embodiment where the training vector V₂ is obtained fromrespective coordinates of feature points of a face, as in the secondembodiment, the sixth embodiment describes obtaining the training vectorV₂ from region images around the coordinates of feature points of aface, and automatically generating a template suitable for amatching-target face image from the matching-target face image.

Specifically, combined information of input sample images and regionimages around coordinates of feature points of a face are divided into aplurality of distributions, correlation between a matching-target imageand coordinates of feature points of the matching-target image isstudied for each distribution, and using the correlations, coordinatesof feature points of the matching-target image are obtained.

Further, the processing of data matching apparatus according to thesixth embodiment is divided broadly into offline processing and onlineprocessing, as in the second and fifth embodiments. Herein, differencesfrom the first embodiment are particularly explained with reference toFIGS. 24 to 25.

The processing of S2400 to S2402 and S2404 in the offline processing inthe sixth embodiment is the same as that of S10 to S12 and S15illustrated in FIG. 6 as in the first embodiment, and descriptionsthereof are omitted. Further, the processing of S2405 to S2408 is thesame as that of S2105 to S2108 in FIG. 21, and descriptions thereof areomitted.

Second training vector calculating section 201 in learning section 103fetches images of respective peripheral regions of portions of a face tocalculate the training vector V₂ (S2403).

Next, after confirming N inputs are finished in S2404, learning section103 executes the processing of S2405 to S2408 shown in FIG. 24, andstores average vectors M₁ ^(k) and M₂ ^(k) and feature extraction matrixC₃ ^(k)* corresponding to Rth distribution in the correlationinformation database. In addition, feature extraction matrix C₃ ^(k)* isproduced from region images on the peripheries of coordinates of featurepoints of the face.

The online processing in the sixth embodiment will be described belowwith reference to an operation flow diagram shown in FIG. 25.

The processing of S2500, S2501 and S2505 to S2507 in the onlineprocessing in the sixth embodiment is the same as that of S20, S21 andS24 to S26 in FIG. 9 as in the first embodiment, and descriptionsthereof are omitted.

Parameter calculating section 302 calculates the expectation vector Efrom the vector X calculated in S2501 using Eq.11 (S2502).

Template matching section 303 generates template images that are modeledto element distributions with R distributions (S2503), performs templatematching, and calculates coordinates of feature points in the secondelectronic data (S2504).

Subsequent processing, S2505 to S2507, is the same as in the firstembodiment, and descriptions thereof are omitted.

As described above, according to the sixth embodiment, sample images aredivided into a plurality of distributions, correlation between thesample images and respective images of portions of the images arestudied for each distribution, and using the correlations, respectiveimages of portions of an input matching-target image are estimated. Itis thereby possible to accurately estimate each portion even whenfluctuations exist on distributions i.e. characteristics of input imagesfor matching. Then, by performing the template matching using theestimated respective images of portions, it is possible to detectcoordinates of feature points of a matching-target input image.

SEVENTH EMBODIMENT

The seventh embodiment copes with the case where fluctuations exist onthe distributions of input vectors that are input sample images, as inthe sixth embodiment. In contrast to the fifth embodiment where thetraining vector V₂ is obtained from respective coordinates of featurepoints of a face and to the sixth embodiment where the training vectorV₂ is obtained from region images around coordinates of feature pointsof a face, the seventh embodiment describes obtaining the trainingvector V₂ from combined vectors of coordinates of feature points of aface and region images around the coordinates of the feature points.Then, coordinates of feature points of a matching-target face image areestimated from thus obtained training vectors V₂, and a templatesuitable for the matching-target face image is automatically generatedfrom the matching-target face image. Using the automatically generatedtemplate with peripheral regions of coordinates of feature points,matching is performed.

It is thereby possible to limit a search area of the template, andperform the matching processing using a sheet of template. As a result,it is possible to greatly reduce cost as compared to the case ofsearching the entire region a matching-target face image using aplurality of templates.

As in the third and fifth embodiments, the processing of the datamatching apparatus according to the seventh is divided broadly intooffline processing and online processing. Herein, particularly,differences from the first are explained with reference to FIGS. 26 and27.

In the offline processing in the seventh embodiment, processing of S2600to S2602 and S2605 is the same as that of S10 to S12 and S15 shown inFIG. 6, and descriptions thereof are omitted.

Second training vector calculating section 201 in learning section 103fetches images of respective peripheral regions of portions in the firstelectronic data using coordinates of feature points input in S2602(S2603), and calculates the training vector V₂ that is a combined vectorof the coordinates of feature points input in S2602 and the imagesfetched in S2603 (S2604).

Next, after confirming inputs corresponding N samples are finished inS2605, learning section 103 executes processing the same as theprocessing of S2405 to S2408 shown in FIG. 24, and stores averagevectors M₁ ^(k) and M₂ ^(k) and feature extraction matrixes C₃ ^(k)*corresponding to R distributions in the correlation information database(S2606 to 2609).

In addition, the feature extraction matrix C₃ ^(k)* is produced from thecombined vector of coordinates of feature points in the first electronicdata and region images on the peripheries of coordinates of featurepoints.

The online processing in the seventh embodiment will be described belowwith reference to an operation flow diagram shown in FIG. 27.

The processing of S2700, S2701 and S2705 to S2707 in the onlineprocessing in the seventh embodiment is the same as that of S20, S21 andS24 to S26 in FIG. 9 as in the first embodiment, and descriptionsthereof are omitted.

Parameter calculating section 302 calculates the expectation vector Efrom the vector X calculated in S2701 using Eq.11 (S2702).

Template matching section 303 generates template image that are modeledto element distributions with R distributions (S2703), performs templatematching on the peripheries of the coordinates of feature pointsobtained from the vector E in S2703, and calculates coordinates offeature points in the second electronic data (S2704).

Subsequent processing, S2705 to S2707, is the same as in the firstembodiment, and descriptions thereof are omitted.

As described above, according to the seventh embodiment, sample imagesare divided into a plurality of distributions, correlation between thesample images and images of respective portions of the images arestudied for each distribution. Then, using the correlations, it ispossible to estimate respective coordinates of feature points in a faceof an input matching-target image are estimated. Further, byhierarchical search processing such that the matching is performed onperipheral regions of the estimated coordinates of feature points usingthe estimated template, it is possible to estimate coordinates offeature points of an matching-target image.

In this way, it is possible to improve the precision in estimatingcoordinates of feature points, and further it is possible to perform theestimation using a sheet of template while restricting a range fortemplate matching to limited regions. As a result, computation cost canbe reduced as compared to the conventional method of searching allareas.

EIGHTH EMBODIMENT

The eighth embodiment of the present invention copes with the case wheresample input images are of vehicle, and input matching-target imageshave various distributions when fluctuations exist on the distributionsof input vectors that are the input sample images of vehicles as in thefifth embodiment.

The processing of data matching apparatus according to the eighthembodiment is divided broadly into offline processing and onlineprocessing, as in the fourth and fifth embodiments. Herein, differencesfrom the first embodiment are particularly explained with reference toFIG. 28.

Since the offline processing of the eighth embodiment is the same asthat in the fifth embodiment, and descriptions thereof are omitted. Inaddition, average vectors M₁ ^(k) and M₂ ^(k) and feature extractionmatrix C₃ ^(k)* for each of R distributions are generated from a vehicleimage, region image of number plate portion, and region image of frontportion.

The online processing in the eighth embodiment will be described belowwith reference to an operation flow diagram shown in FIG. 28.

The processing of S2800, S2801 and S2803 to S2807 in the onlineprocessing in the eighth embodiment is the same as that of S20, S21 andS23 to S26 in FIG. 9 as in the first embodiment, and descriptionsthereof are omitted.

Parameter calculating section 302 calculates the expectation vector Efrom the vector X calculated in S2801 using Eq.11 (S2802).

Matching section 107 calculates a vehicle image, region image of thenumber plate portion and region image of the front portion fromcoordinates of feature points calculated in S2803 (S2804), reads numberson the number plate (S2805), and performs matching on the vehicle imageand number plate (S2806).

As described above, according to the eighth embodiment, sample vehicleimages are divided into a plurality of distributions, correlationbetween the sample vehicle images and images of respective coordinatesof feature points of the images are studied for each distribution, andusing the correlations, images in the respective coordinates of featurepoints of a matching-target vehicle image are estimated.

It is thereby possible to accurately estimate respective coordinates offeature points even when a matching-target vehicle has fluctuations onthe distribution i.e. characteristics. Then, by performing templatematching using the estimated images of respective coordinates of featurepoints as templates, it is possible to detect coordinates of featurepoints of the matching-target vehicle image.

In addition, while in the eighth embodiment the data matching apparatusin the fifth embodiment is applied to a vehicle image matchingapparatus, it may be possible to apply the data matching apparatus insixth or seventh embodiment to a vehicle image matching apparatus.

Further, the present invention is not limited to the above-mentionedembodiments.

The present invention is capable of being carried into practice readilyusing an independent computer system, by reading a program in anothercomputer and storing the program in a storage medium such as a flexibledisk to transfer.

The present invention includes in its aspect computer program productsthat are storage media including instructions for use in programming forcomputers that implements the present invention. The storage mediainclude, for example, flexible disk, optical disk, CDROM, magnetic disk,ROM, RAM, EPROM, EEPROM, magnetic-optical card, memory card and DVD, butare not limited to the foregoing.

The parameter estimation apparatus of the present invention may be usedto match images other than face images and vehicle images.

While in the above-mentioned embodiments the parameter estimationapparatus is applied to image matching, it may be possible to use theparameter estimation apparatus of the present invention as matchingapparatuses for matching other than image matching, by inputting dataother than images as input data. For example, it may be possible toapply the parameter estimation apparatus to a speech matching apparatusby inputting speech data as input data.

This application is based on the Japanese Patent ApplicationsNo.2001-291620 filed on Sep. 25, 2001 and No.2002-129428 filed on Apr.30, 2002, entire contents of which are expressly incorporated byreference herein.

INDUSTRIAL APPLICABILITY

As described above, according to the present invention, it is possibleto accurately obtain coordinates of feature points of an input imagewith less processing cost.

1. A parameter estimation apparatus comprising: an image input thatconverts optical data into electronic data; a parameter input thatinputs coordinates of a first feature point from first electronic datainput from the image input; a learning section that calculatesauto-correlation information from a plurality of items of firstelectronic data, calculates cross-correlation information from theplurality of items of first electronic data and coordinates of aplurality of first feature points, calculates a feature extractionmatrix for estimating coordinates of a second feature point of secondelectronic data input from the image input using the auto-correlationinformation and the cross-correlation information, and outputs a firstaverage vector calculated from the plurality of items of firstelectronic data, a second average vector calculated from the coordinatesof the plurality of first feature points and the feature extractionmatrix; a correlation information database that stores the first averagevector, the second average vector and the feature extraction matrix fromthe learning section; and a parameter estimator that estimatescoordinates of the second feature point, using the second electronicdata, the first average vector, the second average vector and thefeature extraction matrix.
 2. The parameter estimation apparatusaccording to claim 1, wherein the learning section comprises: a firsttraining vector calculator that calculates a first training vector ofvector pattern where the first electronic data is arranged in the orderof scanning; a second training vector calculator that calculates asecond training vector of vector pattern where the coordinates of thefirst feature points are arranged sequentially; an average vectorcalculator that calculates the first average vector by averaging aplurality of first training vectors and calculates the second averagevector by averaging a plurality of second training vectors; anauto-correlation information calculator that receives the plurality offirst training vectors and the first average vector, and calculates apseudo inverse matrix of a first covariance matrix that isauto-correlation information of the first training vectors; across-correlation information calculator that receives the plurality offirst training vectors, the first average vector, the plurality ofsecond training vectors and the second average vector, and calculates asecond covariance matrix that is cross-correlation information of thefirst training vectors and the second training vectors; and a featureextraction matrix calculator that receives the pseudo inverse matrix ofthe first covariance matrix and the second covariance matrix, andcalculates the feature extraction matrix from the pseudo inverse matrixof the first covariance matrix and the second covariance matrix.
 3. Theparameter estimation apparatus according to claim 1, wherein theparameter estimator comprises: a vector calculator that receives thesecond electronic data, and calculates an input vector of vector patternwhere the second electronic data is arranged in the order of scanning; aparameter calculator that calculates an expectation vector including thecoordinates of the second feature point from the input vector, the firstaverage vector, the second average vector and the feature extractionmatrix; and a template matcher that calculates the coordinates of thesecond feature point from the expectation vector.
 4. A parameterestimation apparatus comprising: an image input that converts opticaldata into electronic data; a parameter input that inputs an image aroundcoordinates of a first feature point from first electronic data inputfrom the image input; a learning section that calculatesauto-correlation information from a plurality of items of firstelectronic data, calculates cross-correlation information from theplurality of items of first electronic data and images aroundcoordinates of a plurality of first feature points, calculates a featureextraction matrix for estimating an image around coordinates of a secondfeature point of second electronic data input from the image input usingthe auto-correlation information and the cross-correlation information,and outputs a first average vector calculated from the plurality ofitems of first electronic data, a second average vector calculated fromthe images around the coordinates of the plurality of first featurepoints, and the feature extraction matrix; a correlation informationdatabase that stores the first average vector, the second average vectorand the feature extraction matrix input from the learning section; and aparameter estimator that estimates coordinates of the second featurepoint, using the second electronic data, the first average vector, thesecond average vector and the feature extraction matrix.
 5. Theparameter estimation apparatus according to claim 4, wherein thelearning section comprises: a first training vector calculator thatcalculates a first training vector of vector pattern where the firstelectronic data is arranged in the order of scanning; a second trainingvector calculator that calculates a second training vector of vectorpattern where the images around the coordinates of the first featurepoints are arranged sequentially; an average vector calculator thatcalculates the first average vector by averaging a plurality of firsttraining vectors and the second average vector by averaging a pluralityof second training vectors; an auto-correlation information calculatorthat receives the plurality of first training vectors and the firstaverage vector, and calculates a pseudo inverse matrix of a firstcovariance matrix that is auto-correlation information of the firsttraining vectors; a cross-correlation information calculator thatreceives the plurality of first training vectors, the first averagevector, the plurality of second training vectors and the second averagevector, and calculates a second covariance matrix that iscross-correlation information of the first training vectors and thesecond training vectors; and a feature extraction matrix calculator thatreceives the pseudo inverse matrix of the first covariance matrix andthe second covariance matrix, and calculates the feature extractionmatrix from the pseudo inverse matrix of the first covariance matrix andthe second covariance matrix.
 6. The parameter estimation apparatusaccording to claim 4, wherein the parameter estimator comprises: avector calculator that receives the second electronic data, andcalculates an input vector of vector pattern where the second electronicdata is arranged in the order of scanning; a parameter calculator thatcalculates an expectation vector including the image around thecoordinates of the second feature point from the input vector, the firstaverage vector, the second average vector and the feature extractionmatrix; and a template matcher that calculates the image around thecoordinates of the second feature point from the expectation vector,searches for a matching region that is an image region of the secondelectronic data matching the image around the coordinates of the secondfeature point, and calculates the coordinates of the second featurepoint from the matching region.
 7. A parameter estimation apparatuscomprising: an image input that converts optical data into electronicdata; a parameter input that inputs coordinates of a first feature pointand an image around the coordinates of the first feature point fromfirst electronic data input from the image input; a learning sectionthat calculates auto-correlation information from a plurality of itemsof first electronic data, calculates cross-correlation information fromthe plurality of items of first electronic data and a combined vector ofcoordinates of a plurality of first feature points and images around thecoordinates of the plurality of first feature points, calculates afeature extraction matrix for estimating coordinates of a second featurepoint and an image around the coordinates of the second feature point ofsecond electronic data input from the image input using theauto-correlation information and the cross-correlation information, andoutputs a first average vector calculated from the plurality of items offirst electronic data, a second average vector calculated from aplurality of combined vectors, and the feature extraction matrix; acorrelation information database that stores the first average vector,the second average vector and the feature extraction matrix input fromthe learning section; and a parameter estimator that estimates thecoordinates of the second feature point, using the second electronicdata, the first average vector, the second average vector and thefeature extraction matrix.
 8. The parameter estimation apparatusaccording to claim 7, wherein the learning section comprises: a firsttraining vector calculator that calculates a first training vector ofvector pattern where the first electronic data is arranged in the orderof scanning; a second training vector calculator that calculates asecond training vector of vector pattern where combined vectors arearranged sequentially; an average vector calculator that calculates thefirst average vector by averaging a plurality of first training vectorsand the second average vector by averaging a plurality of secondtraining vectors; an auto-correlation information calculator thatreceives the plurality of first training vectors and the first averagevector, and calculates a pseudo inverse matrix of a first covariancematrix that is auto-correlation information of the first trainingvectors; a cross-correlation information calculator that receives theplurality of first training vectors, the first average vector, theplurality of second training vectors and the second average vector, andcalculates a second covariance matrix that is cross-correlationinformation of the first training vectors and the second trainingvectors; and a feature extraction matrix calculator that calculates thefeature extraction matrix from the pseudo inverse matrix of the firstcovariance matrix and the second covariance matrix.
 9. The parameterestimation apparatus according to claim 7, wherein the parameterestimator comprises: a vector calculator that receives the secondelectronic data, and calculates an input vector of vector pattern wherethe second electronic data is arranged in the order of scanning; aparameter calculator that calculates an expectation vector including thecoordinates of the second feature point and the image around thecoordinates of the second feature point from the input vector, the firstaverage vector, the second average vector and the feature extractionmatrix; and a template matcher that estimates the coordinates of thesecond feature point and the image around the coordinates of the secondfeature point from the expectation vector, searches for a matchingregion that is an image region of the second electronic data matchingthe estimated image around estimated coordinates of the second featurepoint, and calculates the coordinates of the second feature point fromthe matching region.
 10. A parameter estimation apparatus comprising: animage input that converts optical data into electronic data; a parameterinput that inputs coordinates of a first feature point from firstelectronic data input from the image input; a learning section thatdivides a set of combined information of the first electronic data andthe coordinates of the first feature point into a plurality ofdistributions, calculates for each distribution auto-correlationinformation from a plurality of items of first electronic data andcross-correlation information from the plurality of items of firstelectronic data and coordinates of a plurality of first feature points,calculates for each distribution a feature extraction matrix forestimating coordinates of a second feature point of second electronicdata input from the image input using the auto-correlation informationand the cross-correlation information, and outputs the first averagevector calculated from the plurality of items of first electronic dataobtained for each distribution, the second average vector calculatedfrom the coordinates of the plurality of first feature points obtainedfor each distribution, and the feature extraction matrix obtained foreach distribution; a correlation information database that stores thefirst average vector, the second average vector and the featureextraction matrix obtained for each distribution and input from thelearning section; and a parameter estimator that estimates coordinatesof the second feature point for each distribution, using the secondelectronic data, the first average vector, the second average vector andthe feature extraction matrix.
 11. The parameter estimation apparatusaccording to claim 10, wherein the learning section comprises: a firsttraining vector calculator that calculates a first training vector ofvector pattern where the first electronic data is arranged in the orderof scanning; a second training vector calculator that calculates asecond training vector of vector pattern where the coordinates of firstfeature points are arranged sequentially; a distribution elementcalculator that generates a plurality of combined vectors from theplurality of first training vectors and the plurality of second trainingvectors, and using probability distributions of a set of the combinedvectors, divides the first training vectors and the second trainingvectors into a plurality of distributions; an average vector calculatorthat calculates for each distribution the first average vector byaveraging a plurality of first training vectors and the second averagevector by averaging a plurality of second training vectors; anauto-correlation information calculator that receives the plurality offirst training vectors and the first average vector for eachdistribution, and calculates for each distribution a pseudo inversematrix of a first covariance matrix that is auto-correlation informationof the first training vectors; a cross-correlation informationcalculator that receives the plurality of first training vectors, thefirst average vector, the plurality of second training vectors and thesecond average vector for each distribution, and calculates for eachdistribution a second covariance matrix that is cross-correlationinformation of the first training vectors and the second trainingvectors; and a feature extraction matrix calculator that receives thepseudo inverse matrix of the first covariance matrix and the secondcovariance matrix for each distribution, and calculates the featureextraction matrix from the pseudo inverse matrix of the first covariancematrix and the second covariance matrix.
 12. The parameter estimationapparatus according to claim 10, wherein the parameter estimatorcomprises: a vector calculator that receives the second electronic data,and calculates an input vector of vector pattern where the secondelectronic data is arranged in the order of scanning; a parametercalculator that calculates an expectation vector including thecoordinates of the second feature point from the input vector, the firstaverage vector classified for each distribution, the second averagevector classified for each distribution and the feature extractionmatrix classified for each distribution; and a template matcher thatcalculates the coordinates of the second feature point from theexpectation vector.
 13. A parameter estimation apparatus comprising: animage input that converts optical data into electronic data; a parameterinput that inputs an image around coordinates of a first feature pointfrom first electronic data input from the image input; a learningsection that divides a set of combined information of the firstelectronic data and the image around the coordinates of the firstfeature point into a plurality of distributions, calculates for eachdistribution auto-correlation information from a plurality of items offirst electronic data and cross-correlation information from theplurality of items of first electronic data and images aroundcoordinates of a plurality of first feature points, calculates for eachdistribution a feature extraction matrix for estimating an image aroundcoordinates of a second feature point of second electronic data inputfrom the image input using the auto-correlation information and thecross-correlation information, and outputs the first average vectorcalculated from the plurality of items of first electronic data obtainedfor each distribution, the second average vector calculated from theimages around the coordinates of the plurality of first feature pointsobtained for each distribution, and the feature extraction matrixobtained for each distribution; a correlation information database thatstores the first average vector, the second average vector and thefeature extraction matrix obtained for each distribution and input fromthe learning section; and a parameter section that estimates coordinatesof the second feature point for each distribution, using the secondelectronic data, the first average vector, the second average vector andthe feature extraction matrix.
 14. The parameter estimation apparatusaccording to claim 13, wherein the learning section comprises: a firsttraining vector calculator that calculates a first training vector ofvector pattern where the first electronic data is arranged in the orderof scanning; a second training vector calculator that calculates asecond training vector of vector pattern where the coordinates of firstfeature points are arranged sequentially; a distribution elementcalculator that generates a plurality of combined vectors of theplurality of first training vectors and the plurality of second trainingvectors, and using probability distributions of a set of the combinedvectors, divides the first training vectors and the second trainingvectors into a plurality of distributions; an average vector calculatorthat calculates for each distribution the first average vector byaveraging a plurality of first training vectors and the second averagevector by averaging a plurality of second training vectors; anauto-correlation information calculator that receives the plurality offirst training vectors and the first average vector for eachdistribution, and calculates for each distribution a pseudo inversematrix of a first covariance matrix that is auto-correlation informationof the first training vectors; a cross-correlation informationcalculator that receives the plurality of first training vectors, thefirst average vector, the plurality of second training vectors and thesecond average vector for each distribution, and calculates for eachdistribution a second covariance matrix that is cross-correlationinformation of the first training vectors and the second trainingvectors; and a feature extraction matrix calculator that receives thepseudo inverse matrix of the first covariance matrix and the secondcovariance matrix for each distribution, and calculates the featureextraction matrix from the pseudo inverse matrix of the first covariancematrix and the second covariance matrix.
 15. The parameter estimationapparatus according to claim 13, wherein the parameter estimatorcomprises: a vector calculator that receives as its input the secondelectronic data, and calculates an input vector of vector pattern wherethe second electronic data is arranged in the order of scanning; aparameter calculator that calculates an expectation vector including theimage around the coordinates of the second feature point from the inputvector, the first average vector classified for each distribution, thesecond average vector classified for each distribution and the featureextraction matrix classified for each distribution; and a templatematcher that calculates the image around the coordinates of the secondfeature point from the expectation vector, searches for a matchingregion that is an image region of the second electronic data matchingthe image around the coordinates of the second feature point, andcalculates the coordinates of the second feature point from the matchingregion.
 16. A parameter estimation apparatus comprising: an image inputthat converts optical data into electronic data; a parameter input thatinputs coordinates of a first feature point and an image around thecoordinates of the first feature point from first electronic data inputfrom the image input; a learning section that divides a set of combinedinformation of the first electronic data, the coordinates of the firstfeature point and the image around the coordinates of the first featurepoint into a plurality of distributions, calculates for eachdistribution auto-correlation information from a plurality of items offirst electronic data and cross-correlation information from theplurality of items of first electronic data, coordinates of a pluralityof first feature points and images around the coordinates of theplurality of first feature points, calculates for each distribution afeature extraction matrix for estimating coordinates of a second featurepoint and an image around the coordinates of the second feature point ofsecond electronic data input from the image input using theauto-correlation information and the cross-correlation information, andoutputs a first average vector calculated from the plurality of items offirst electronic data obtained for each distribution, a second averagevector calculated from the coordinates of the first feature points andthe images around the coordinates of the first feature points obtainedfor each distribution, and the feature extraction matrix obtained foreach distribution; a correlation information database that stores thefirst average vector, the second average vector and the featureextraction matrix obtained for each distribution and input from thelearning section; and a parameter estimator that estimates thecoordinates of the second feature point for each distribution, using thesecond electronic data, the first average vector, the second averagevector and the feature extraction matrix.
 17. The parameter estimationapparatus according to claim 16, wherein the learning section comprises:a first training vector calculator that calculates a first trainingvector of vector pattern where the first electronic data is arranged inthe order of scanning; a second training vector calculator thatcalculates a second training vector of vector pattern where thecoordinates of the first feature points and the images around thecoordinates of the first feature points are arranged sequentially; adistribution element calculator that generates a plurality of combinedvectors from the plurality of first training vectors and the pluralityof second training vectors, and using probability distributions of a setof the combined vectors, divides the first training vectors and thesecond training vectors into a plurality of distributions; an averagevector calculator that calculates for each distribution the firstaverage vector by averaging a plurality of first training vectors andthe second average vector by averaging a plurality of second trainingvectors; an auto-correlation information calculator that receives theplurality of first training vectors and the first average vector foreach distribution, and calculates for each distribution a pseudo inversematrix of a first covariance matrix that is auto-correlation informationof the first training vectors; a cross-correlation informationcalculator that receives the plurality of first training vectors, thefirst average vector, the plurality of second training vectors and thesecond average vector for each distribution, and calculates for eachdistribution a second covariance matrix that is cross-correlationinformation of the first training vectors and the second trainingvectors; and a feature extraction matrix calculator that receives thepseudo inverse matrix of the first covariance matrix and the secondcovariance matrix for each distribution, and calculates the featureextraction matrix from the pseudo inverse matrix of the first covariancematrix and the second covariance matrix.
 18. The parameter estimationapparatus according to claim 16, wherein the parameter estimatorcomprises: a vector calculator that receives as its input the secondelectronic data, and calculates an input vector of vector pattern wherethe second electronic data is arranged in the order of scanning; aparameter calculator that calculates an expectation vector including thecoordinates of the second feature point and an image around thecoordinates of the second feature point from the input vector, the firstaverage vector classified for each distribution, the second averagevector classified for each distribution and the feature extractionmatrix classified for each distribution; and a template matcher thatcalculates the coordinates of the second feature point and the imagearound the coordinates of the second feature point from the expectationvector, searches for a matching region that is an image region of thesecond electronic data matching the image around the coordinates of thesecond feature point, and calculates the coordinates of the secondfeature point from the matching region.
 19. A data matching apparatuscomprising: an image input that converts optical data into electronicdata; a parameter input that inputs coordinates of a first feature pointfrom first electronic data input from the image input; a learningsection that calculates auto-correlation information from a plurality ofitems of first electronic data, calculates cross-correlation informationfrom the plurality of items of first electronic data and coordinates ofa plurality of first feature points, calculates a feature extractionmatrix for estimating coordinates of a second feature point of secondelectronic data input from the image input using the auto-correlationinformation and the cross-correlation information, and outputs a firstaverage vector calculated from the plurality of items of firstelectronic data, a second average vector calculated from the coordinatesof the plurality of first feature points, and the feature extractionmatrix; a correlation information database that stores the first averagevector, the second average vector and the feature extraction matrixinput from the learning section; a parameter estimator that estimatescoordinates of the second feature point, using the second electronicdata, the first average vector, the second average vector and thefeature extraction matrix; an image data base that stores the firstelectronic data; and a matching calculator that calculates a matchingregion that is an image for use in matching from the second electronicdata and the coordinates of the second feature point, and matches thematching region with the first electronic data stored in the imagedatabase to obtain matching.
 20. A data matching apparatus comprising:an image input that converts optical data into electronic data; aparameter input that inputs an image around coordinates of a firstfeature point from first electronic data input from the image input; alearning section that calculates auto-correlation information from aplurality of items of first electronic data, calculatescross-correlation information from the plurality of items of firstelectronic data and images around coordinates of a plurality of firstfeature points, calculates a feature extraction matrix for estimating animage around coordinates of a second feature point of second electronicdata input from the image input using the auto-correlation informationand the cross-correlation information, and outputs a first averagevector calculated from the plurality of items of first electronic data,a second average vector calculated from the images around thecoordinates of the plurality of first feature points, and the featureextraction matrix; a correlation information database that stores thefirst average vector, the second average vector and the featureextraction matrix input from the learning section; a parameter estimatorthat estimates coordinates of the second feature point, using the secondelectronic data, the first average vector, the second average vector andthe feature extraction matrix; an image database that stores the firstelectronic data; and a matching calculator that calculates a matchingregion that is an image for use in matching from the second electronicdata and the coordinates of the second feature point, and matches thematching region with the first electronic data stored in the imagedatabase to obtain matching.
 21. A data matching apparatus comprising:an image input that converts optical data into electronic data; aparameter input that inputs coordinates of a first feature point and animage around the coordinates of the first feature point from firstelectronic data input from the image input; a learning section thatcalculates auto-correlation information from a plurality of items offirst electronic data, calculates cross-correlation information from theplurality of items of first electronic data and a combined vector ofcoordinates of a plurality of first feature points and images around thecoordinates of the plurality of first feature points, calculates afeature extraction matrix for estimating coordinates of a second featurepoint and an image around the coordinates of the second feature point ofsecond electronic data input from the image input using theauto-correlation information and the cross-correlation information, andoutputs a first average vector calculated from the plurality of items offirst electronic data, a second average vector calculated from aplurality of combined vectors, and the feature extraction matrix; acorrelation information database that stores the first average vector,the second average vector and the feature extraction matrix input fromthe learning section; a parameter estimator that estimates thecoordinates of the second feature point, using the second electronicdata, the first average vector, the second average vector and thefeature extraction matrix; an image database that stores the firstelectronic data; and a matching calculator that calculates a matchingregion that is an image for use in matching from the second electronicdata and the coordinates of the second feature point, and matches thematching region with the first electronic data stored in the imagedatabase to obtain matching.
 22. A data matching apparatus comprising:an image input that converts optical data into electronic data; aparameter input that inputs coordinates of a first feature point fromfirst electronic data input from the image input; a learning sectionthat divides a set of combined information of the first electronic dataand the coordinates of the first feature point into a plurality ofdistributions, calculates for each distribution auto-correlationinformation from a plurality of items of first electronic data andcross-correlation information from the plurality of items of firstelectronic data and coordinates of a plurality of first feature points,calculates for each distribution a feature extraction matrix forestimating coordinates of a second feature point of second electronicdata input from the image input using the auto-correlation informationand the cross-correlation information, and outputs the first averagevector calculated from the plurality of items of first electronic dataobtained for each distribution, the second average vector calculatedfrom the coordinates of the plurality of first feature points obtainedfor each distribution, and the feature extraction matrix obtained foreach distribution; a correlation information database that stores thefirst average vector, the second average vector and the featureextraction matrix obtained for each distribution and input from thelearning section; a parameter estimator that estimates the coordinatesof the second feature point for each distribution, using the secondelectronic data, the first average vector, the second average vector andthe feature extraction matrix; an image data base that stores the firstelectronic data; and a matching calculator that calculates a matchingregion that is an image for use in matching from the second electronicdata and the coordinates of the second feature point, and matches thematching region with the first electronic data stored in the imagedatabase to obtain matching.
 23. A data matching method, comprising:converting optical data into electronic data; inputting coordinates of afirst feature point from first electronic data of the convertedelectronic data; calculating auto-correlation information from aplurality of items of first electronic data, calculatingcross-correlation information from the plurality of items of firstelectronic data and coordinates of a plurality of first feature points,calculating a feature extraction matrix for estimating coordinates of asecond feature point of second electronic data of the convertedelectronic data using the auto-correlation information and thecross-correlation information, and outputting a first average vectorcalculated from the plurality of items of first electronic data, asecond average vector calculated from the coordinates of the pluralityof first feature points, and the feature extraction matrix; storing thefirst average vector, the second average vector and the featureextraction matrix in a correlation information database, and estimatingcoordinates of the second feature point using the second electronicdata, first average vector, the second average vector and the featureextraction matrix; and storing the first electronic data in an imagedatabase, calculating a matching region that is an image for use inmatching from the second electronic data and the coordinates of thesecond feature point, and collating the matching region with the firstelectronic data stored in the image database to obtain matching.
 24. Adata matching method, comprising: converting optical data intoelectronic data; inputting an image around coordinates of a firstfeature point from first electronic data of the converted electronicdata; calculating auto-correlation information from a plurality of itemsof first electronic data, calculating cross-correlation information fromthe plurality of items of first electronic data and images aroundcoordinates of a plurality of first feature points, calculating afeature extraction matrix for estimating an image around coordinates ofa second feature point of second electronic data of the convertedelectronic data using the auto-correlation information and thecross-correlation information, and outputting a first average vectorcalculated from the plurality of items of first electronic data, asecond average vector calculated from the images around the coordinatesof the plurality of first feature points, and the feature extractionmatrix; storing the first average vector, the second average vector andthe feature extraction matrix in a correlation information database, andestimating the coordinates of the second feature point using the secondelectronic data, the first average vector, the second average vector andthe feature extraction matrix, the second electronic data, the firstaverage vector, the second average vector and the feature extractionmatrix; and storing the first electronic data in an image database,calculating a matching region that is an image for use in matching fromthe second electronic data and the coordinates of the second featurepoint, and collating the matching region with the first electronic datastored in the image database to obtain matching.
 25. A data matchingmethod, comprising: converting optical data into electronic data;inputting coordinates of a first feature point and an image around thecoordinates of the first feature point from first electronic data of theconverted electronic data; calculating auto-correlation information froma plurality of items of first electronic data, calculatingcross-correlation information from the plurality of items of firstelectronic data and a combined vector of coordinates of a plurality offirst feature points and images around the coordinates of the pluralityof first feature points, calculating a feature extraction matrix forestimating coordinates of a second feature point and an image around thecoordinates of the second feature point of second electronic data of theconverted electronic data using the auto-correlation information and thecross-correlation information, and outputting a first average vectorcalculated from the plurality of items of first electronic data, asecond average vector calculated from a plurality of combined vectors,and the feature extraction matrix; storing the first average vector, thesecond average vector and the feature extraction matrix in a correlationinformation database, and estimating coordinates of the second featurepoint, using the second electronic data, the first average vector, thesecond average vector and the feature extraction matrix; and storing thefirst electronic data in an image database, calculating a matchingregion that is an image for use in matching the second electronic dataand the coordinates of the second feature point, and collating thematching region with the first electronic data stored in the imagedatabase to obtain matching.
 26. A data matching method, comprising:converting optical data into electronic data; inputting coordinates of afirst feature point from first electronic data of the convertedelectronic data; dividing a set of combined information of the firstelectronic data and the coordinates of the first feature point into aplurality of distributions, calculating for each distributionauto-correlation information from a plurality of items of firstelectronic data and cross-correlation information from the plurality ofitems of first electronic data and coordinates of a plurality of firstfeature points, calculating for each distribution a feature extractionmatrix for estimating coordinates of a second feature point of secondelectronic data of the converted electronic data using theauto-correlation information and the cross-correlation information, andoutputting the first average vector calculated from the plurality ofitems of first electronic data obtained for each distribution, thesecond average vector calculated from the coordinates of the pluralityof first feature points obtained for each distribution, and the featureextraction matrix obtained for each distribution; storing the firstaverage vector, the second average vector and the feature extractionmatrix obtained for each distribution, and estimating coordinates of thesecond feature point for each distribution, using the second electronicdata, the first average vector, the second average vector and thefeature extraction matrix; and storing the first electronic data in animage database, calculating a matching region that is an image for usein matching from the second electronic data and the coordinates of thesecond feature point, and collating the matching region with the firstelectronic data stored in the image database to obtain matching.
 27. Acomputer readable medium for storing a computer program for instructinga computer to execute data matching processing, the computer readablemedium comprising: an inputting code segment for inputting coordinatesof a first feature point from first electronic data of electronic dataconverted from optical data; a calculating code segment for calculatingauto-correlation information from a plurality of items of firstelectronic data, calculating cross-correlation information from theplurality of items of first electronic data and coordinates of aplurality of first feature points, calculating a feature extractionmatrix for estimating coordinates of a second feature point of inputsecond electronic data input from the auto-correlation information andthe cross-correlation information, and outputting a first average vectorcalculated from the plurality of items of first electronic data, asecond average vector calculated from the coordinates of the pluralityof first feature points, and the feature extraction matrix; anestimating code segment for storing the first average vector, the secondaverage vector and the feature extraction matrix in a correlationinformation database, and estimating coordinates of the second featurepoint using the second electronic data, the first average vector, thesecond average vector and the feature extraction matrix; and a storingcode segment for storing the first electronic data in an image database,calculating a matching region that is an image for use in matching fromthe second electronic data and the coordinates of the second featurepoint, and collating the matching region with the first electronic datastored in the image database to obtain matching.
 28. A computer readablemedium for storing a computer program for instructing a computer toexecute data matching processing, the computer readable mediumcomprising: an inputting code segment for inputting an image aroundcoordinates of a first feature point from first electronic data ofelectronic data converted from optical data; a calculating code segmentfor calculating auto-correlation information from a plurality of itemsof first electronic data, calculating cross-correlation information fromthe plurality of items of first electronic data and images aroundcoordinates of a plurality of first feature points, calculating afeature extraction matrix for estimating an image around coordinates ofa second feature point of second electronic data of the convertedelectronic data using the auto-correlation information and thecross-correlation information, and outputting a first average vectorcalculated from the plurality of items of first electronic data, asecond average vector calculated from the images around the coordinatesof the plurality of first feature points, and the feature extractionmatrix; an estimating code segment for storing the first average vector,the second average vector and the feature extraction matrix in acorrelation information database, and estimating coordinates of thesecond feature point using the second electronic data, the first averagevector, the second average vector and the feature extraction matrix; anda storing code segment for storing the first electronic data in an imagedatabase, calculating a matching region that is an image for use inmatching from the second electronic data and the coordinates of thesecond feature point, and collating the matching region with the firstelectronic data stored in the image database to obtain matching.
 29. Acomputer readable medium for storing a computer program for instructinga computer to execute data matching processing, the computer readablemedium comprising: an inputting code segment for inputting coordinatesof a first feature point and an image around the coordinates of thefirst feature point from first electronic data of electronic dataconverted from optical data; a calculating code segment for calculatingauto-correlation information from a plurality of items of firstelectronic data, calculating cross-correlation information from theplurality of items of first electronic data and a combined vector ofcoordinates of a plurality of first feature points and images around thecoordinates of the plurality of first feature points, calculating afeature extraction matrix for estimating coordinates of a second featurepoint and an image around the coordinates of the second feature point ofsecond electronic data of the converted electronic data using theauto-correlation information and the cross-correlation information, andoutputting a first average vector calculated from the plurality of itemsof first electronic data, a second average vector calculated from aplurality of combined vectors, and the feature extraction matrix; anestimating code segment for storing the first average vector, the secondaverage vector and the feature extraction matrix in a correlationinformation database, and estimating coordinates of the second featurepoint using the second electronic data, the first average vector, thesecond average vector and the feature extraction matrix; and a storingcode segment for storing the first electronic data in an image database,calculating a matching region that is an image for use in matching fromthe second electronic data and the coordinates of the second featurepoint, and collating the matching region with the first electronic datastored in the image database to obtain matching.
 30. A computer readablemedium for storing a computer program for instructing a computer toexecute data matching processing, the computer readable mediumcomprising: an inputting code segment for inputting coordinates of afirst feature point from first electronic data of electronic dataconverted from optical data; an calculating code segment for dividing aset of combined information of the first electronic data and thecoordinates of the first feature point into a plurality ofdistributions, calculating for each distribution auto-correlationinformation from a plurality of items of first electronic data andcross-correlation information from the plurality of items of firstelectronic data and coordinates of a plurality of first feature points,calculating for each distribution a feature extraction matrix forestimating coordinates of a second feature point of second electronicdata of the converted electronic data using the auto-correlationinformation and the cross-correlation information, and outputting thefirst average vector calculated from the plurality of items of firstelectronic data obtained for each distribution, the second averagevector calculated from the coordinates of the plurality of first featurepoints obtained for each distribution, and the feature extraction matrixobtained for each distribution; an estimating code segment for storingthe first average vector, the second average vector and the featureextraction matrix obtained for each distribution in a correlationinformation database, and estimating coordinates of the second featurepoint for each distribution using the second electronic data, the firstaverage vector, the second average vector and the feature extractionmatrix; and a storing code segment for storing the first electronic datain an image database, calculating a matching region that is an image foruse in matching from the second electronic data and the coordinates ofthe second feature point, and collating the matching region with thefirst electronic data stored in the image database to obtain matching.